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I. iNTRODUCTiON 

this report describes the results o£ several 
studies conducted during the second implementation . 
o£ the individualized,- multimedia Leadership, 
Psfychoiogy and Management course developed by- 
West inghouse Learning Corporation (WLC) for the 
United States Naval Academy (USNA) . These studies 
form part o£ a comprehensive investigation o£ factors 
infiy^hcinj: student achieveinent intended to guide 
cbntintiing improveme^^^^ the course and to have 
wide ranging application in the field of educational 
technology. / 

The research- conducted during the fall 1970 
semester basically was a replication of the research 
efforts engaged in during the first implementation 
of the course in the spring semester of 1970. This 
report will provide a brief description of the initial 
'research plan which was reported in detail in TR-6.12a 
Report of Phase IT Research Findings, Part I; Conditions 
of Instruction , the major emphasis will be directed 
toward an explication of the modifications to the 
research design for the second implementation, and 

comparison, where applicable, of the results of the 
experimental manipulations in the spring and fall 
implementations of the course. 
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The WLC research plan has the distinction of 
being one of the first to provide a joint examination 
of factors in all major categories relevant to the 
desr.gn of an instructional system, including media, 

t 

presentation forms, task requirements, student 
chaTacteristics, and operationial- organization. The 
WLC plan is unique bpth in this number of factors 
investigated and in the use of an entire ongoing 
course system as an experimental vehicle permitting 
-empiricai findings to be extracted relevant to the 
influence" of each factor singly and in combination. 
It is expected that experiments of this type, as part 
of a concentrated effort in educational research, 

may eventually result in a comprehensive understanding 

■■( 

of the educational process, so that an-instructor 
may choose with confidence the most effective media 
and presentation forms to teach, a particular type of 
student a particular type of task. 



.II. BACKGROUND OF THE RESEARCH 

In this section an informal analysis of educational 
systems is presented leading to the id.entificiation of 
major classes of variables important in the development 

■ i 

of a comprehensive educational technology. An important 
distinction between presentation and media variables, is 
then described in relation to conceptual organization of 
these variable classes, based on Tpsti and Ball's (1969) 
instructional design and media selection model. 
Vafiabies in Ihstructiohal Systems 

Instructional systems may be analyzed as an 
interactive process among components of (1) designed 
behavioral objectives, (2) student, (3) materials 
designed to change student behavior toward the designed 
objectives, (4) media presenting materials to the 
student, and (5) operational organization bringing these 
components, into articulation.' The nature of such a 
system is illustrated in Figure 1. 
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An analysis of this kind clarifies several 'points 
in x:elation to instructional systems. First, evaluation 
of a system rests primarily on examination of the 
correspondence between objectives desired and 
behavioral changes achieved. Other c'*'iteria, such as 
the desirability of objectives and cost-effectiveness 
are external to the system and concern the utility of 
the system as a component in larger social systems. 
Second, research on instructional systems requires the 
manipulation or measurement of characteristics (variables) 
differing among individual examples of the same 
component, rnd study of the associated modifications 
in the behavioral output. Clearly, evaluation of a 
standing instructional. system may proceed, given information 
on objectives and behavior change, but when a discrepancy 
is discovered between these, efforts must return to the 
research domain to discover how to modify the 
system to eliminate the discrepancy. 

Corresponding to each component of the instructional 
,system are a large number of variables potentially 
important in determining the final behavior change 
effected. The instruction system designer is, however, 
not equally free to manipulate or select settings for 
the variables in all categories. Once objectives for 



a given targt^t population of students are established 
tor the system, the range of variation in task variables 
is , fixed, by the objectives, and student variables are fixed 
by the definition of the target population. The systen de-. 
signer must then work with the presentation variables of the 
materials, media variables, and operational variables to 
arrange an optimally effective system for the particular 
types of tasks and students involved* From this point 
of view, presentatioii, media, and operational variables 
are of jprimary research interest, while task and student 
variables are of interest mainly in relationship to the 
other categories of variables* 

Informal analyses of this type have provided 
heuristic guidelines in the development of WLC's research 
plan, and should prove useful in similar efforts in the 
future • 

Presentation .Variables 

One of the important questions in current educational 
research is, "Which media will tfiach a given unit of 
instructional material most effectively?" With the 
introduction of so many different technological aids 

(including teaching machines, programed texts, television, 

» * 

film, "cartridge tape, and computerized instruction) there 
is a steadily increasing variety of devices available 



for use in any instructional program. The problem is 



to decide what medium is best for a particular purpose, 

and how to efficiently, utilize^^^ts capabilities* 

In addressing^this- problem, Tosti and Ball (1969) 

have developed a model for instructional design and 

media selection in which a distinction is made between 

medium and presentation: 

Media researchers. to date have not 
. chosen to distinguish a presentation 
form from the media wh carry i^. 
the lieW^ model requires that such a 
sep^aratibh^ be * Aiade . 

The media in instructional systems 
carry not only the data of the 
instructional message, but also 
data on students* responses aiid 
various bits of data necessary to 
maintain the operating systems. 
It is this conglomerate of 
information carried by a medium 
which will be called the presentation. 

Presentation forms will be explicitly 
structured to communicate all data 
(stimulus, response i system control, 
student control) necessary for an 
efficient student- system interface. 
A student does not learn from the 
media. He learns from the. presentation 
form. Media do little more than 
deliver the information to be learned 
in whatever presentational form 
previously decided upon. Some media 
organizations have maintained that 
media choice may contribute to 
learning efficacy because of a 
student's media preference char- 
acteristics or because of media 
dependent cues. However, the im- 
portance of these two ideas is 
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minimal when a separate presentation 
design is implemented. 

To illustrate the distinction that Tosti and Ball 
have made 9 consider an instructor giving a lecture to 
one group, of students. This lecture is videotaped and 
later shown to another group of. students. Both groups 
would experience the same presentation design; only 
the medium would be different* 

Also consider one lecture in which the instructor 
never answers questions, and another lecture where the 
instructor answers every question. In this case, the 
med^i are the same (lecture), but the presentation 
of the two lectures is different. 

The instructional design model whic^ Tosti and 
Ball have developed is essentially a taxonomy of 
instructional presentation variables, independent of 
media device, content, and external constraints. Using 
this taxonomy, it becomes possible to precisely describe 
any instructional sequence by identifying its characteristics 
along basic dimensions which are common to all instructional 
presentations. Since the specif icatiou of presentational 
variables is a critical consideration in educational 
research, the application of Tosti and Ball^s model 
may result in a significant improvement in the quality of 



studies in educational technology and in the generaliz- 

ability of their findings. The presentation taxonomy 

may be of comparable value to other educational 

classification systems. As Bloom (1956) commented • * 

on the taxonomy of education objectives: 

••.(the taxonomy) is expected to be of 
general help to all teachers, administrators, 
professional specialists, and research workers 
who deal with curricular arid eyaiuation problems. 

is especially intended to help them discuss 
these problems with greater precision^ 

The application of the Tosti and Ball model involves 

the determination of a precise presentation design for 

each instructional objective. Media are then selected on 

the basis of their limitations in presenting the presentation 

design intact. The primary question raised by the Tosti 

and Ball model is whether variations of conditions of 

instruction in the presentation design domain are of 

greater or lesser importance than variation in the media 

domain with respect to student achievement. . The implication 

is that if the presentation design is held constant over 

a unit of instruction the use of different media should not 

result in differential levels of achievement. Conversely 

variations in the presentation design with the medium 

constant over a unit of instruction should result in 

differential levels of achievement. 
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It would appear that a detailed behavioral analysis 
of individual learning events would require the specifications 
of a host of subsidiary variables associated with each of 
the Tosti and Ball presentation variables. The general 
approach used here was to investigate the presentation 
design--media controversy over larger units of instruction 
involving similar types of- instructional objectives. 

In each of the hypotheses to be tested during 
the Leadership Management course, the experimental 
treatments have been defined with reference to the six 
dimensions of presenta1:iori of Tosti and Ball's model. 
The dimensions of presentation are discussed in detail 
in the following section. 
Dimensions of , Presentation 

Dimensions of presentation have been derived by a 
logical analysis of instructional systems (Tosti and 
Ball^ 1969). These- systems possess three basic 
capabilities: 

a. The transmission of instructional information 
(stimulus capability) 

b. Accepting measurable behavior of the student 
(response capability) 

c. '^' Changing the presentation based on the behavior 

of the student (management capability) 
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Each capability may be further differentiated in 
terms of two attributes: form and frequency. The result 
of this analysis is a 3 x 2 matrix, represented in Table 1, 
in which six dimensions of presentatitin are generated. 
Further study of common or possible instructional procedures 
reveals a number of levels or categories associated with 
each dimension, also listed in table 1. 

The following subsections give a detailed description 
9f each dimension and its levels or categories. Discussions 
of research findings relating to the dimensions will be 
presented in connection with the experiment: involving 
those dimensions. 

» 

Stimulus form (representation) This dimension 
is related most directly to media. It characterizes 
the dominant mode of sensory reception (by the student) 
of the instructional material, inherent in the means of 
representation of stimuli. There are three categories 
within this dimension: 

a. Verbal -written written material, such as 
printed text 

b. Verbal -spoken -- voice transcriptions, such 
as from a lecturer, videotaped lecture,, -or 
tape recorder 
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TABLE "l 

SYSTEM CAPABILITY MATRIX AND 



DIMENSIONS OF PRESENTATION^ 



System 
Capabfllty 


Attributes of System Capabi 1 Ity 


Form 


Frequency 


Stimulus 


Stimulus Representation 


Duration 


Verbal -spoken 
Pictorial 


1 ra n s 1 enr-rers 1 sTa nt 

Length of time the 
presentation remains 
Intact 

a. low*. 

b. Intermediate 
f c. high 


' Response 


Response Demand 


Response-Demand Frequency 


Overt-written . 

Overt-spoken 

Covert 


1 nf requent-Frequent 

Frequency of response 
required 

a. low or zero 

b. Intermediate 

c. high 


Management 


Management Form 


Management Frequency 


Repetition 
Multi-level 
Multi-form 
Error-diagnostic 


1 nf requ6nt-Frequent 

Frequency of decision 
to change presentation 

a. low or zer^o 

b. Intermediate 

c. high 



a 

After Tost? and Ball, 1969. 
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c. Pictorial.-- illustrative material, such as 
pictures 

In many presentations, two or more stimulus forms 
may be used simultaneously. A book may display both 
illustrations and prose. An educational television 
program conveys both a picture and lecture. Other 

•presentations can require media-mixes such as the 
teacher-blackboard combination. Less common is the 
simultaneous employment of two variations of the same 
stimulus presentation, i.e., requiring the student to 

.read and listen to the same verbal presentation. 

Stimulus frequency (duration) . Tosti and Ball 

(1969) have explained stimulus frequency as follows: 

Presentation varies on this ordinal dimension 
from transient to persistent, depending upon 
the duration of the simulus. Movies usually 
are conveyors of more transient presentation, 
and texts display relatively persistent ones. 
A classroom presentation by lecture is more 
transient than one which is delivered by the 
blackboard. 

Transient presentations are usually instructor controlled. 
As in most films and lectures, the stimuli are available 
to the student for a fixed period of time. Persistent 
presentations are usually student controlled. An example 
would be the PI text, in which the student proceeds at 
his own rate and may ^study a unit of instructional 
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material for as long as he likes. Essentially, a 
presentation form is categorized as "transient" or 
"persistent" depending on the length of time the 
presentation stimuli remain unchanged. 

Response demand . This dimension characterizes 
the types of behaviors which students are expected to 
perforin in an instructional situation. The four 
categories within this dimension are: 

a. Covert 

b. Overt-written 

c. Overt -vocal 

d. Passive 

In a PI text, the student is asked to write the 
answers to small units of materials. This presentation 
design has an overt-written response demand. A student 
who is asked a question in a group discussion usually 
answers in the overt -vocal form. The covert category 
describes situations where the student is asked a 
question, but is not required to answer with a specific, 
overt (observable) response. For example, the instructor 
giving a lecture might say, "Think of what would happen 
if we mixed sodium and water." The passive category 
describes those presentations in which questions are not 
overtly asked, and the student is not expected to respond 



with specific overt or covert responses. Many lectures 
and most films are in this category. The student is only 
required to look and 'listen. However, the label "passive" 
should not imply that the student is doing nothing; he 
may be thinking intently, formulating questions about 
the material, or taking notes. This behavior, however, 
is controlled by the student, not by the instructional 
material. Where it is the intent of the. instruction to 
evoke relatively specific behaviors in the learner, that 
presentation is categorized as either overt-written, 
overt- spoken, or covert. 

Response demand frequency . This dimension describes 
ho\ frequently the student is expected to respond 
(overtly or covertly) in a given period of instruction. 
A PI text normally has a response demand after every 
frame. A lecture of film may be presented with no 
response demands in the entire session (or module) . In 
any medium, questions or problems may be interposed at 
various intervals during the instructional sequence. 
This presentation design would have some intermediate 
response demand frequency. This variable may provide 
*a better conceptualization of what has been termed 
"step size" than any other. 

In a temporal sequence of instruction, there are 
three general dimension categories: 
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a« High response demand frequency relatively 
frequent demand for a response in aii 
instructional sequence ^ such as in programed 
texts where a response is required in each 
frame • 

b* Medium response demand frequency relatively 
moderate frequency of demand for a response, 
such as questions which follow ten minutes of 
videotaped lecture. 
c« Low response demand frequency low demand for 
a response 9 as when a "review" question follows 
a chapter of textual material. 
' Management form > . Instructional management 
can be defined as those activities involved in the decision 
to assign a specific le|irning exercise to a given student ^ 
based on the assessment of some behavior of that student. 
One common example of instructional management occurs 
when the teacher » who discriminates that a student is 
having difficulty with learning a particular skilly makes 

the decision to assign special homework or decides to pro* 

J*- 

vide individual tutoring. The general logic of this 
activity^ i^e.p assessing behavior ^ selecting presentation^ 
and. then having the student engaging in new activity, can 
be extended to provide the foundation for rules employed 
in most new individualized instructional systems and 
computer -managed^ classroom programs. 
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Every instructional system involves three 
management elements: « 

a. Repertoire assessment *- appraisal of data 
and analysis of behavior competencies 

b. Selection decision selection of a goal as 
a result of decisions based on assessment 

c. Activity actions following from decision. 
It is evident that the elements of instructional 

management can vary in their composition, depending on 

the purpose of management. Tosti and Ball (1965)} 

isolated five purposes that may be achieved.^ These are: . 

a. Need management --to ensure students receive 
only those materials which they require to 
meet their objective. 

b. Achievement management --to ensure all students 
have mastered the objectives of the segment. 

c. Prescription management --to ensure a given 
student receives the most appropriate materials 
to meet the objectives, in terms of his 
individual characteristics. 

d. Motivation management --to ensure continual 
student contact with the materials and to 

^Tosti and Ball (1969) originally identified a sixth type, 
that of operational management. In the current piesentation, 
this category is included among the operational system vari- 
ables, since the management activities are rarely contingents 
on assessment of an individual student *5 performance. 
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► 

increase student learning rate* 
e« Enrichment management to provide for 

additional information relevant to objectives » 
but not necessary for their achievement. 
The present research is primarily concerned with 
achievement management. Therefore, the four categories 
in the Management Type dimension presented below are 
the different procedures which may be used in 
management for achievement. 

It frequently happens that a. student is not 
responding to the presentation in a manner which allows 
him to reach the objectives. There are four strategic 
subclasses o,f management responses to such situations. 

a. Repetition --If the student fails to reach 
the objective, repeat the same presentation 
or continue through similar presentations 
until he does. Continuous practice is 

one variation of this strategy. 

b. Multiform -- If the student fails to reach 
the objective with one presentation form, ' 
select a parallel but different form, e.g;. 
Project PLAN (Flanagan, 1967). 

c. Multilevel --If the student fails to reach 
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the objectives with the presentation form, 
select a lower level (more expanded) form 
e.g., PROMOD (C'de Baca, 1968). 
d. Error-Diagnostic If an error is made at 
any point within the presentation, action 
designed to correct that specific error is 
selected, e.g., intrinsic program presentatipn 
or computer assisted instruction (CAI) 
presentation. It is necessary when using 
the error -diagnostic strategy to classify 
errors as: 

1) input errors due to poor presentation 
design. 

2) processing errors due to the student's 
lack of the assumed appropriate repertoire 
on which the learning material was built, 

> or the student's use of an inappropriate 

approach to the solution. 

3) output errors -- due to carelessness, poor 
attention, and chance error (failed to' 
attend to a significant stimulus) . 

Management frequency . "This dimension is ordinal 
and is ordered according to the relative frequency of 
the decision to modify the presentation" (Tosti and 
Ball, 1968). The concept of dec is ion -making in pres- 
entation design is more clearly exemplified in tutoring. 
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Typically, the student is directed to answer a question 
posed by the tutor, and a decision is made by the tutor 
about what he should next present, on the basis of that 
response. A similar instructional management form is used 
in PI. If an answer is incorrect, the student may be di- 
rected to any one of a number of remediation frames. 

Other media may also vary in decision frequency An 
instructor may ask his class a question in the middle of 
his lecture to see if they are understanding the material . 
Depending upon the students' answers, the instructor may 
decide to continue with the planned lecture, to review the 
same material, or to start a new topic. For any presenta- 
tion form, the decision frequency may vary from a decision 
every frame to no decision at all. 

In a temporal sequence of instruction, there are 
three general categories; 

a. High management frequency relatively high 

frequency of decision to alter the presentation, 
based on the. student's response to a question. 
Management frequency may be built into the' 
instructional system, as in a text where the 
decision is made on the basis of a response 
to every frame or to remediate him on the 
same frame. The management frequency may 
also be determined extemporaneously, as when 
a lecturer asks a class a question; if no one 
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answers, the lecturer may decide to review 
previous content. 

b. Medium management frequency — relatively 
moderate frequency of decision to alter the 
presentation based on the student's response, 
such as having a quiz after a 10-minute film, 
and on the basis of the student's score, 
either repeating^ the film or proceeding to 
rew material. ^ 

c. Ldw management frequency relatively ^ low 
frequency of decision to alter instruction 
based on the student's response to a 
question, such as a lecturer giving a quiz 
after 40 minutes of lecture; basing the 
decision on the student's score, the 
instructor either assigns homework problems 
or does not. i 

It should be noted that the response-demand frequency 
must be equal to or more than the management frequency; de- 
cisions about a response cannot be made more frequently than 
one calls for that response. An example of a presentation 
in which response-demand frequency exceeds management frequency 
is the lecturer who frequently asks the class "rhetorical 
questions"; the lecturer does not change his presentation on 
the basis of the student's (covert) responses, yet he does 
call for those responses. In this case the response-demand 
frequency would be high but the management frequency would 
be low. (See Table 1) 
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Task Variables 

The conditions of instructi' n investigated 
were designed to affect the processes of learning 
in a comprehensive fashion, altering the instruction 
related to every objective of a segment in specified 
ways. The main question raised by the classifications 
of types of learning is whether or not the 
presentation variables and media havesimilar 
effects^ on the achievement of different types of 
objectives. 

Early in WLC analysis of content and objectives 
for the USNA Leadership course, it became apparent 
that most of the terminal objectives of the course 
represented the types of learning (tasks) indicated 
by Gagne (1965) to be principles and problem 
solving with enabling objectives at the levels of 
verbal associations, multiple discrimination, and 
concepts in Gagne 's hierarchy of learning tasks. 
It was also recognized that most of the elements 
involved in these objectives could be further 
analyzed as rules and examples in accordance with 
Evans, Homme, and Glasor'r> (1962) RULEG system. 
Furthermore, the objectives could be further 
identified according to Bloom's (1956) taxonomy 
as involving either knowledge of leadership and 
management elements or application of those 
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elements in realistic naval situations. 

As a result of these findings, WLC developed a 
system of formats to be used as guidelines in the writing 
of specific enabling and terminal objectives. This 
classification scheme represents a derivation and 
extension of Bloom's (1956) Taxonomy, Gagne's (1965) 
learning types, and Evans, Homme, and Glaser's (1962) 
RULEG system, serving to coordinate features of each. 

"Behavioral objectives were prepared, in most 
cases, in accordance with the four formats listed below: 

a. Type 1 (Definition - Identification) 

Given the instruction to identify the correct 
(purpose of/definition of/description of/use of) 
concept X, the M will select from several choices 
the correct (purpose of/definition of/description 
bf/use of) concept .X. 

b. Type 2 (Discrimination-- Comparison) 
Given the instruction to evaluate the 

relationship between/defining attributes of/ ' 
contrast between/comparison among) classes 
X, Y, Z...N, the M will be able to select from 
several choices the paragraph (which illustrates/ 
describes/differentiates/ identifies) this 
(relationship between defining attributes of 
contrast between/comparison among) classes X, 
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Type 3 (Generalization • Problem Identification) 

1) Deductive 

Given examples of X, the M will be able to 
select from several choices the example 
which illustrates principle Y. 

2) Inductive 

Given an example of X, the M will be able to 
select from several choices which principle 
(X, Y, or N) is (shown/exemplif ied/ 
demonstrated) by the example, 
d. Type 4 (Problem Solving) 

When asked to evaluate a situation which is an 
example of class X, the M will select from 
several choices the correct (solution/approach/ 
method/resolution of/reaction to) the situation 
using principle Y. 
Test items were developed from the statement of 
objectives for the criterion-referenced Progress Check 
tests and Administrative test which served as direct 
measures of achievement for purposes of evaluation.- 
Performance on these tests thus represents learning of 
all four types given above, when the instructional 
materials are developed to explicitly teach those objectives 

The primary variables representing different classes 
of learning, however, were the Cumulative Posttest (CPT) 
items developed as special norm-referenced research tests. 
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These tests were developed to have approximately equal 
numbers of items representing acquisition of knowledge 
(Type I items), and applica^ajmrdf knowledge (Type II items), 
roughly corresponding to Bloom's categories of knowledge 
and applications* As ^tems designed for content validity 
with high discriminative power, both types of items tap 
abilities in Bloom's other categories of comprehension, 
analysis, synthesis and evaluation. 

Operationally, the distinction between Type I and 
II CPT items rests almost entirely on the presence or absence 
of naval situational examples in the stems or distractors. 
Thus there are some items which do not correspond precisely 
to Bloom's distinction between knowledge and applications. 
The use of this operational criterion of distinction, 
however, appeared compelling after a careful analysis of 
content represented in the behavioral objectives. 

The specific question raised in the present re- 
search is whether media and presentation variables have 
similar effects: (1). averaged over specific criterion 
behavior of all types as indexed by the Progress Checks, 
and (2) on criterion-related behavior involving compre- 
hension, analysis, synthesis, and evaluation of knowledge 
and the application of knowledge as measured in the CPT 
tests. 
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III. THE RESEARCH PLAN 

Course Structure 

The Leadership Management course was first organized 
in terms of elemental blocks of content and related tests of 
student achievement, which were temporally sequenced without 
regard to research constraints. Additional elements of 
structure were then inserted for research purposes. This 
procedure insured that a basic course structure was 
achieved from which the research elements could easily be 
detached for purposes of final course packaging and 
implementation. The course structure may be described 
in terms of the four categories outlined below. 

Part . The content is divided into 12 parts, 
corresponding to 12 chapters of the basic content outline. 
Each part is a formal designation of a large topic area, 
representing a substantial number of closely related 
terminal objectives relatively independent of the objectives 
of other parts. The objectives of any one part could- be 
considered to be subsumed under one of the broad aims 
(macro-objectives) of the course. The part served -primarily 
as an aid in fractionating the developmental work on 
materials. 

Segment . In terms of content, a segment is a sub- 
collection of learning objectives within a part, which 
are closely related in the development of a behavioral 
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hierarchy of competence and in the sequencing of instruc- 
tional events. A total of 59 segments were incorporated 
in the 12 parts of the course. The content headings of 
each segment are listed in Table 2 under their respective, 
parts* 

Except for revisions based on data obtained from 
the first implementation (Spring 1970), the content of the 
parts and segments remained the same for the second implemen- 
tation (Fall 1970). There were some changes in medium in 
which some of the segments were programed « These changes 
from the spring 1970 to the fall 1970 run are indicated in 
Table 2. A more complete description of the media used in 
each of the two runs can be found in TR6.11, Phase il 
Evaluation Report and TR6.15, Phase III Evaluation Report . 
Changes to the research from the spring to the fall run are 
indicated in Table 2, and discussed in detail in the next 
section (Experimental Design) of this report. 

Conceived operationally^' the segment is the basic 
instructional unit in the development and production of 
materials, and serves as the logistical unit in implementation 
for purposes of scheduling and assessment of progress 
through the course materials. Essentially, the segment 
is analogous to a class period or lesson in other 
instructional systems, requiring 40 to 80 minutes of 
student time, and provides the basis for manipulation of 
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TABLE 2 

OUTLINE OF COURSE STRUCTURE AND MEDIA 



Spring Fal I Spring Fal I 

Part & 

Segment Content Heading CPT CRT ■ 

•Number Unlta Unlta Medium'^ Medium** 

PART ONE: OVERVIEW OF LEADERSHIP 



1 1 


Concepts of Leadership 


MP * 


KIP 


CT 
o 1 


CT 




dTanuaras OT LGaOerSfiip in Tne INaVai oGrVICG 


KID 


KtD 

NK 








rrs\\ InUi liMUi V lUUAL Dtn/\VIUK 












inTroaucTion to rsycnoiogy 


KID 


K 


CT 

ol 


CT 

ST 




Denavior ana its UDSGrvoTion 


t 
1 


1 


AT- or 


AT OD 

AT-Pd 










V 1 — rD 






Learning 


t 
1 


t 


AT A*» 

Ai- or 


AT OD 

AT-Pd 










VT-PB 




2.4 


. Factors Affecting Learning 


1 


. 1 


AT- or 


AT-PB 










\IT OD 

VT-Pd 




Z« ? 


oTTenTion ana rercepTion 


t 
1 


t 
1 


AT— A*» 

Ai- or 


AT OD 

A 1 -Pd 










llT OD 

vT-Pd 




z«o 


Ma 4* I t/m4> t An 

MOT 1 vaTion 




Z 


CT 

ST 


^T 

ST 


O 7 
/ 


pAn4 1 t a4> 

uont 1 1 CT 


Z 


Z 


CT 

51 


^T 

ST 


z«o 


NeuroTic ana rsycnoTic KeacTions 


Z 


Z 


CT 

bl 


CT 

ST 




DAf^f Ana t T '4*i# 

rersona i i Ty 


KJD 


R 


t AC 
LAo 


CT 

ol 




DAPT TMPPP* f^DAMD nVKIAMIPC 










^« 1 


v^naracTer 1 5T 1 C5 ot oroups 




\ 


AT- or- 

A 1 — or 


AT—DP 
A 1 — rD 








V \ — rD 






1 n6 r\6 1 OT 1 ons>n 1 p ot Tn6 L6au6r to Tne oroup 




X 


AT- rir- 

A 1 or 


AT-DP 
A 1 — rD 








VT-PB 




3.3 


Group Interactions 


3 


3 


AT- or 


AT-PB 










VT-PB 




3.4 


Conformity as a Factor of Group Behavior 


3 


3 


AT- or 


AT-PB 










VT-PB 




3.5 


Relation of the Individual to the Group 


NR 


R 


ST 


ST 




PART FOUR: ACHIEVING EFFECTIVE COMMUNICATION 










4.1 


Importance of Interpersonal Communication 


4 


4 


LT 


LT 


4.2 


Types of Communication 


4 


4 


LT 


LT 


4.3 


The Communication Process 


4 


4 


LT 


LT 




(Receiver and Barriers) 










4.4 


The Communication Process 


5 


5 


AT- IP 


AT- IP or 




(Sender and Feedback) 








ATS- IP 


4.5 


Formal Communication and Its Dimensions 


5 


5 - 


AT- IP 


AT- IP or 












ATS- IP 


4.6 


Informal Communication 


5 


5 


AT- IP 


AT- IP or 












ATS- IP 


4.7 


Communication Under Battle Situations 


5 




AT- IP 


AT- IP or 



ATS- IP 
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Spring Fa 1 1 Spring Fa 1 1 

Part & 

Segment Content Heading CPT CPT 

Number Unlt^ Uhlt^ Medium** Medium** 

PART FIVE: MILITARY MANAGEMENT 



5.1 


In+roducMon to Mdndgement and th6 Management 


NR 


14 


ST 


ST 




Process 










5.2 


Decision Making and Creativity 


NR 


14 


ST 


ST 


5.3 


Objectives 


NR 


14 


ST 


ST 


5.4 


Planning 


6 


6 


LT 


LT 


5.5 


Organizing: Principles and Process 


6 


6 


LT 


LT 


5.6 


Organ l2l ng: Structure 


6 


6 


LT 


LT 


5.7 


Organ Izl ng: ChartI ng 


7 


7 


AT- or 


AT-PB 








VT-PB 




5.8 


Directing 


7 


7 


AT- or 


AT-PB 








VT-PB 




5.9 


Control 1 Ing 


7 


7 


AT- or 


AT-PB 








VT-PB 




5.10 


Coordinating 


7 


7 


AT- or 


AT-PB 








VT-PB 






PART SIX: AUTHORITY AND RESPONSIBILITY. 










6.1 


Concept of Authority 


8 


8 


ST 


ST 


6.2 


Why Peppje Accept/Resist Authority 


8 


8 


ST 


ST 


6.3 

we ^ 


DelAnatlon Authorltv* Line-Staff 


8 


8 


ST 


ST 




Relationship 










€.4 


Responsibility 


NR 


R 


ST 


ST 




PART SEVEN: LEADERSHIP BEHAVIOR AND STYLE 










7.1 


Leadership Behavior • 


9 


9 


AT- or 


AT-PB 










VT-PB 




7.2 


Leadership Style 


9 


9 


AT- or 


AT-PB 








VT-PB . 




7.3 


Determiners of Leadership Style - The Leader 


9 


9 


AT- or' 


AT-PB 








VT-PB 




7.4 


Determiners of Leadership Style - The Group 


9 


9 


AT- or 


AT-PB 




and The Situation 






VT-PB 




7.5 


Participative Leadership 


NR 


R 


VT-PB 


AT-PB or 












ATS-PB 




PART EIGHT: SENIOR-SUBORDINATE RELATIONSHIPS 










8<l 


Organizational Structure & Social Distance In 


10 


10 


LT 


LT 




Senior-Subordinate Relationships 










8.2 


Officer-Enlisted Relationships 


10 


10 


LT 


LT 


8.3 


Assumption of Command and Formal & Informal 


10 


10 


LT 


LT 




Leader. Relationships 










8.4 


Introduction to Counseling 


II 


II 


LAS 


ST 


8.5 


The Counseling Process 


II 


II 


LAS 


ST 


8.6 


Relations with Seniors and Contemporaries 

* 


II 


II 


LAS 


ST 




PART NINE: MORALE - ESPRIT DE CORPS 










9.1 


Morale 


NR 


R 


VT-PB 


AT-PB or 



ATS-PB 



i 



Part & 

Segment 

Number 



Spring Fal I 



• 30 

Spring 



Fal 



Content Heading 
PART NINE: MORALE - ESPRITDE CORPS (CON»T) 



CPT CPT . 
Unit* Unlta Medium** Medium'* 



O 9 


oroup doiiuQiiiy ano cspriT 


KIP 


p 


V l-ro 


AI**rD or 












ATQ-PP 
A i ^••rD 




PART TPM* ni^riPl IMP 










in 1 


1 nTrOvjucT 1 on TO uiscipiino 


KID 


p 


AT*. ID 


AT— 1 D 

Al— Ir or 










AT^- 1 P 


10-2 




KIP ' 


p 




AT- IP rsr 

A 1 1 r or 












AT^- 1 P 
A i O** i r 




PART ELEVEN: PERSONNEL EVALUATION 










M.i 


The Role of Evaluation 


12 


12 


ST 


ST 


ii.2 


Enlisted Performance Evaluation 


12 


12 


ST 


ST 


ii.3 


Officer Evaluation 


12 


12 


ST 


ST 




PART TWELVE: APPLIED LEADERSHIP 










12.1 


^ Measurement of Effective LeadershI;; 


13 


13 


CAI 


CAI or 












CAIS 


12.2 


Generally Recognized Characteristics of an 


13 


13 


CAI 


CAI or 




Effective Leader 








CAIS 


12.3 


Techniques of Assuming Command 


13 


13 


CAI 


CAI or 










CAIS 


12.4 


"That's an Order!" 


13 


13 


CAI 


CAI or 












CAIS 



NR refers to a nonresearch segment, thus not 
.R refers to a research segment that did not 



b 



ST 


= Syndactic (multi-level ) Text 


LAS = 


F-GD 


= Film, Group Discussion 
= Audiotape 


LT = 


AT 


IP = 


ATS 


= Audiotape Script 


CAI = 


VT 


= Videotape 


CAIS = 


PB 


= Panel Book 





assigned to a CPT unit* | 

Involve a Cumulative Posttest. % 

I 

Learning Activities Summary 

Linear Text ;j 

Intrinsic Program 1 

Computer Assisted Instruction 4 

Computer Assisted Instruction Script | 
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the rfeal-time parameters of the course. 

At the completion of each segment , a progress check 
(PC) test was administered to assess the student *s attain < 
ment of the terminal and enabling objectives of the 
segment. PC's are composed o2 10 criterion-referenced 
items, developed directly from the behavioral statement 
of segment objectives. 

Module . A module is a particular instructional 
condition used to prepare and deliver materials for a 
segment, i4entified in terms of the categories of the 
Tosti and Ball (1969) model. Several parallel modules 
were prepared in each segment utilized for research 
purposes, representing variations specified by the 
experimental designs. The different modules of a segment 
are distinguishable from one another by differences in 
presentation design and/or media, although the content 
is the same. Specifications of the modules for each 
segment are outlined in later sections of the paper 
giving the design of each experiment. 

Cumulative Posttest unit . The cumulative posttest 
(CPT) unit is a group of three or four adjacent segments 
within a part. In the Spring 1970 run, there were 13 CPT 
units involving 45 of the 59 segments of the course, as 
listed in Table 2. The primary criteria for grouping 
segments into CPT units were that the segments dealt with 
similar types of content and objectives, and that the 
instructional sequences relating to particular concepts 
which were initiated in the unit would also terminate in 
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the same unit. All segments in a OPT unit were developed 
in the same medium and ^ith the same variartions in 
instructional conditions between modules. 

In the second implementation (Fall 1970), there 
were 14 CPT units involving 48 of the 59 segments* In 
addition, research using progress checks as the dependent 
measure was conducted involving^ 9 additional segments. 

The CPT unit is the fundamental unit of instruction 
for research purposes, providing the framework on which 
the experimental designs were constructed. The students 
were divided into groups assigned to different modules 
in the CPT unit. A student in any one group would thus 
encounter the same experimental conditions in progressing 
through the three segments of the unit, and would 'take 
three PC^s, one after completing his module of each segment 
After completing the segments and PC^s all students then 
take t» ^ CPT, a test administered to assess overall achieve 
ment level under the experimental conditions represented 
in the CPT unit. . 

Performance on the CPT was the primary dependent 
measure for research purposes. Each CPT was composed of 
10 multiple-choice items for each segment in the unit, 
so that CPT*s for 3 segment units had 30 items, and CPT*s 
for 4 segment units had 40 items. There were approximately 
equal numbers of two typ^s of items: Type I, representing 



33 

acquisition of knowledge of the concepts and principles 
in the unit, and Type II, representing application of those 
concepts and principles in the unit in relation to 
realistic examples of leadership situations. 

CPT items were designed to have content validity 
in relation to the objectives of the unit, but unlike 
the PC items, also to have high difficulty and discrimina- 
tion power. The CPT tests thus provided norm-referenced 
rather than criterion-referenced measures of achievement 
level. Many items were designed to measure the ability 
to integrate behaviors from different segments in the 
unit. An effort was made, however, to maintain an 
equitable representation of content from the several 
segments of the unit. 

Following completion of the CPT, each student is 
given remediation on segments where his PC test performance 
is below 80t. The remediation consists of repetition 
of the same instructional materials previously used with 
the segment, or materials of an alternative module thought 
to be more effective. On completing remediation, the 
student repeats the PC*s for those segments and then 
proceeds to the next segment. 

Media . Course mterials were prepared using 
eight media: 

, a. audiotaped lectures with accompanying panelbook 
(AT-PB) 
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b* audiotape script with accompanying panelbook 
(AT>PB) 

c. syndactic text summary statement on concepts 
^ in text followed by frame sequence in text (ST) 
d» audiotape with intrinsically programed text (AT- IP) 
e, audiotape script with intrinsically programed 

text (ATS- IP) 
f» computer assisted instruction (CAI) 
g» computer assisted instruction script (CAIS) 
h» linear programed text (LT) 
i. film and group discussion (F-GD) 

The nature of these media are explained in more detail 
in the Phase III Evaluation Report (TR6.15). 
Experimental Design 

Spring 1970 > Table 3 presents a summary of the 
research plan for the spring 1970 run of the course. 
Experiment I involved 16 segments in which two variables 
were manipulated. Each student worked through two con- 
secutive CPT units in videotape and two consecutive CPT 
units in audiotape. Half of the students had high RDF 
presentations and half had low RDF presentations. Experiment 
II involved nine segments in which two variables Were 
being manipulated with the medium of linear text being used 
in all segments. Each of the students worked with each 
of the three types of response demand (i*e», covert in 
CPT 4, overt-spoken in CPT 6 and overt-selected in CPT 10) 
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in different sequences. Half of the students were given 
high RDF presentations across all segments and half were 
given low RDF presentations. Experiment III covered 
eight segments in which two variables were manipulated. 
Each student was presented only one of the four conditions 
or modules in CPT 5 and 13 and it was the same one in 
both CPT units. As indicated in Table 3, the medium 
used in CPT unit 5 was audiotape/intriniscally programed 
booklet, and in CPT unit 13 the medium was Computer- 
Assisted Instruction. Experiment IV involved 9 segments 
in which the remediation method was varied. Each student 
worked through one CPT unit in each of the three conditions 
Finally, Experiment V covered three segments in which peer 
interaction as opposed to individual study was investigated 

The reader should refer to TR6.12a Phase II Research 
Findings, Part I: Conditions of Instruction for a detailed 
discussion of these experiments and their outcomes. 

Fall 1970 . The changes made to the research plan 
for the Fall run are indicated in Tables 4 and 5. In 
Experiment I the use of videotape was dropped. The new 
variables that were manipulated were the form of the respon 
(RD) and provision of confirmation (knowledge of results). 
In Experiment II the ovett-spoken RD type was replaced with 
one that required the midshipmen to underline their respons 
In the Spring run, performance where Computer-Assisted 
Instruction (CAI) was used was exceptionally high and there 
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was some question whether it ;^ras due to the medium itself 
or to the content programed for it. Therefore, an additional 
condition was added to Experiment III. This condition 
involved the use of a script version of the CAI and audiotape/ 
IP presentations. In Experiment IV the conditions remained 
the same, but the students were forced through the remedia- 
tion sequence which they encountered. Experiment Y saw 
the greatest change. -The LAS units were re-written in 
Syndactic text format, the use of the content map was 
investigated, and another CPT unit (involving 3 additional 
segments) was added. 

Some questions were added to the overall research 
plan that involved the use of the progress check data rather 
than the CPT data. These subsidiary questions involved the 
use of nine more segments in the research effort as is shown 
in Table 5. In the first analysis the variables of advance 
organizers and the use of a special revealed answer form 
were manipulated. Each student saw each of the four conditions. 
The question had been raised whether a paper version would 
be as effective as the hardware bound versions (i.e., CAI 
and audiotape) . Therefore, in segment 7.5, in addition to 
the use of the advance organizer, the effectiveness of the 
audiotape script as opposed to the tape itself was studied. 
The third subsidiary analysis looked at the same variables 
as the second but it included the use of the type with the 
intrinisically programed booklet as well as the panelbook. 
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Each student saw each of the four basic conditions, two 
with the panelbook and two with the intrinsic booklet. 

The final difference between the two implementations 
dealt v;ith the dependent measures used. In the spring 
run, the data were analyzed with respect to type I 
and type II CPT scores as well as Total CPT scores. 
In addition, the analyses were conducted using 
progress check performance as a dependent measure. 
In the fall run only total CPT scores were used in 
the major experiments. Progress check data were used, 
however, in the subsidiary analyses. 
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IV, RESEARCH IMPLEMENTATION 

Students 

Forty- four third classmen (sophomores) from the 
United States Naval Academy were enrolled in the Leader- 
ship, Psychology and Management course. 

Before commencing work on the course materials, 
each student was randomly assigned to a track. This 
student track indicated the precise module of materials 
a student would use in each of the 59 segments of the 
course. 
Materials 

The basic course structure was discussed in 
Chapter III, and Table 2 presented an outline of the 
course structure and the media used. The- segments listed 
in Table 2 are core segments. That is, they are 
required segments which include all of the information 
pertinent to the attainment of the requisite behavioral 
objectives. In addition to core segments there were 
depth core and enrichment segments. Depth core segments 
were associated with one or more segments and were 
directed toward amplifying the learning objectives of 
those segments. Depth core segments included in the 
second implementation were film, group discussions, and 
classroom lectures by the USNA instructor. Unlike core 
segments, depth core were scheduled by the instructor 
with respect to time and place. Student attendance was 
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required. Enrichment segments were related to but not 
essential to the mastery of terminal objecr.ives.* . They 
were optional to students who desired more information 
than that presented in core segments. 

The specific media used for each segment are 
indicated in Table 2. A description of the media and each 
of the variations in presentation form (modules) within 
each medium is given in Sections V through X. 
Tests 

Four different tests were used throughout the 
course. They were the administrative pretest and post- 
test, the progress check, the cumulative posttest (CPT) , 
and the USNA examination. 

The administrative pre and posttest was an 80 
point criterion referenced test composed of items 
representatively sampled from the objective-test item 
pool. There was at least one administrative test item 
for each segment of the course. The pretest was given 
at the beginning of the course, and the posttest was 
given as part of the final examination. 

The progress check was a criterion referenced 
test of approximately ten items. It was given at the 
end of each segment. 

The cumulative posttest (CPT) was a norm 
referenced research test composed of positively 
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discriminating content-related test items. Each CPT 
was composed of ten items for each segment in the unit. 
Cumulative posttests were given at the end of each 
experimental unit. 

USNA examinations were a combination of criterion 
referenced test items selected from the objective-test 
item pool and items developed by the USNA on-site 
instructor. These were the only tests in the course 
which were used to determine the midshipmen's grades. 
Procedures 

The second implementation of the course was 
conducted in the Fall of 1970. The course was administered 
by the USNA on-site instructor, the WLC on-site instructor, 
and a course administrator. Detailed procedures used in 
implementing the course are given in the Instructor's 
Guide (TR6.6). 

The instructor's basic responsibilities were 
tutoring students needing remediation, leading group 
discussions, scheduling and administering depth core 
segments, scheduling and administering examinations^ 
and determining grades. 

The course administrator developed and supervised 
the logistical procedures of the course, controlled 
dissemination and collection of all core materials, 
remediation prescription forms, module questionnaires, 
progress checks, and cumulative posttests (CPTs) . The 
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course administrator also scored progress checks and 
CPTs and forwarded data to WLC's computer center. 

Students were routed through the course according 
to procedures outlined in the Student Guide (TR6.5). 
In brief, students worked through core segments of the 
course at their own speed. They were allowed to check 
out materials and study them whenever and wherever they 
wished. All students were given identical material when 
they studied a non-research segment; i.e., they were 
instructed by the same form of presentation. For research 
segments, they studied by the particular module (form of 
presentation) to which they were assigned. Students were 
randomly assigned to modules at the beginning of the course. 
Each student received his own routing schedule which 
included not only the sequence of segments he must study 
but also the schedule for rt^mediation, research tests, 
and USNA examinations. 

Students worked through non-research material by 
studying a segment, taking a progress check, remediating 
(if necessary), and then retaking the progress check. 
The requirement for remediation was based on failure 
to attain 80% of the objectives as jneasured by the 
progress check. If the student failed to meet the 80% 
criterion on his first try, he was given a remediation 
prescription form which directed him to specific points 
in the materials which related to the objectives failed. 



If the student failed to meet the 80% criterion 

following remediation, he reported to the on-site 

instructor for tutoring. 

Students worked through research segments in the 

same manner as non-research segments except that they 

did not remediate until after they had completed the 

entire research unit and taken the cumulative posttest. 

Specific procedures followed in the research segments 

arc discussed in sections V through X. 

Facilities 

For the implementation of the course, WLC was 
provided three classrooms at the Naval Academy. One 
room which was designated as the administrative office 
contained desks for the administrative staff and 
storage space for half of the course materials (including 
tapes, printed material, tests, forms, and computer 
cards) . The administrative room was used as the site 
for administrative conferences, for student tutoring, 
and for distribution and collection of all material. 

The second room was used as the principal 
instruction room. It contained 15 student carrels. 

The third room; used as a multi-purpose room, 
had three carrels to handle overflow from the instructional 
room. In addition, there were 30 student writing desks 
which were used during depth core lectures, films, 
group discussions, and testing. 
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V. EXPERIMENT I 

Introduction 

The puipose of Experiment I was to evaluate the 
effects of variation in response demand frequency (RDF) 
on student achievement with transient media. Two 
additional variables investigated were the effects of 
variation in the form of the response demand (RD) and 
the effects of provision of confirmation. 

RDF was defined earlier as "how frequently the 
student is expected to respond (overtly or covertly) 
in a given period of instruction." It is the relative 
rate of response which is specifically elicited by the 
presentation design. In most programed instruction, the 
student is directed to respond in each frame (relatively 
high RDF). A lecture may proceed through an entire 
class period without requiring the students to answer 
a specific question (relatively low RDF). 

It should be restated that the RDF is independent 
of the management frequency and mantigeiaent type dimensions. 
The answer to a question may or may not lead to a decision 
to change the presentation form, and an incorrect response 
may or may not be followed by remediation. An example 
of a presentation design with high RDF, but low decision 
frequency and no remediation, would be a film in which 
verbal -written questions are interposed at various 
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intervals. Students would be directed to "think of the 
answer" to each question, and feedback (correct answer) 
might be given immediately. However, the film wou:id 
continue regardless of what the student's answers were. 

The primary hypothesis tested in this experiment 
was whether transient presentations with high RDF will 
be more effective than transient presentations with 
low RDF. The present hypothesis is important for two 
practical reasons. First, it is difficult or impossible 
to have a high individual decision frequency with some 
forms of basically transient media (such as television, 
film, videotape, or audiotape). The use of these media 
could be greatly extended if it was shown that a high 
RDF by itself produces superior performance. Secondly, 
in most presentation designs a question is usually the 
occasion for a decision and for some kind of feedback. 
OTet feedback and remediation after the answer may be 
less important than the occurrance of the question per 
se. In a well constructed presentation design, the 
inclusion of appropriate questions at optimal intervals 
may be the most critical factor in producing superior 
performance. 
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Method 

Design and Analysis > The plan of experiment I 
is presented in Table 6. The design is a repeated 
measures half -plaid Latin Square. The students were 
randomly assigned to the four groups. Two of these 
groups responded overtly in each of the four "experimental 
combinations of response demand frequency ^nd provision 
of confirmation. The other two groups responded 
covertly in each of the four experimental conditions. 

Table 6 
Plan of Experiment I 

Cumulative Post Test Unit 



Group 


1 


3 


7 


9 


0 1 

V 

E 


High RDF 
Confirm 


Low RDF 
No Confirm 


Low RDF 
Confirm 


High RDF 
No Confirm 


R 2 
T 


Low RDF 
No Confirm 


High RDF 
Confirm 


High RDF 
No Confirm 


Low RDF 
Confirm 


C 3 

0 

V 


Low RDF 
Confirm 


High RDF 
No Confirm 


High RDF 
Confirm 


Low RDF 
No Confirm 


E 4 

R 

T 


High RDF 
No Confirm 


Low RDF 
Confirm 


Low RDF 
No Confirm 


High RDF 
Confirm 



Materials and Procedures , The segments involved in 
this jexperiment were 2.2 - 2.5, 3.1 - 3.4, 5.7 - 5.10, and 
7.1 - 7.4, and the medium used was audiotapes with panel - 
'* books (see Table 2). 
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Two basic modules were prepared; they varied 
in the response demand frequency (RDF) dimension of 
the presentation design. 

One module of the videotape lecture was 
characterized by high response demand frequency. At 
appropriate points in the tape, the student was 
referred to a numbered question in the panelbook. 
Students read the question and recorded their answers 
on an answer sheet if they were in the overt response 
condition, otherwise they simply thought the answer. 
If the student was in a module where confirmation 
was provided, he would check his answer and then proceed 
with the next section of the tape. The number of 
questions asked ranged from 15 to 22. The low response 
demand frequency module of the audiotape lecture was 
developed by simply editing out all but three of the 
references to questions in the panelbook. All other 
elements of the audiotape lecture remained the same. 

The lecturer for the audiotapes was a commercial 
radio announcer. The lecturer presented all materials 
verbatum from a prepared script. All audiotapes were 
developed in a commercial recording facility. In 
developing the audio presentations, standard recording 
tapes were used. For student use, tapes were transferred 
to C-60 and C-90 cassette cartridges. All charts. 
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Results 

Mean student performance as a function of 
response demand frequency, response demand and provision 
of confirmation is given in Table 7. 

Table 7 

Mean Percent Correct Responses for Conditions of Instruction 

In Experiment I 



High RDF 
Low RDF 



Confirmation 



65.91 



65.75 



No Confirmation 



69.54 



65.91 



67.72 
65.83 



65.83 



67.72 



Overt RD 
Covert RD 



Confirmation 



67.98 



63.68 



No Confirmation 



67.39 



68.05 



67.69 
65.8.6 



High RDF 
Low RDF 



Overt Rr 
68.78 



66.67 




Although there \vas a consistent 21 difference 
between the levels of each of the main effects, the 
analysis of variance (see Appendix a ) showed no 
significant main effects for the form of the response, 
the response demand frequency, or provision of confirmation. 
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There was a tendency for the effects of RDF, 
confirmation and RDF x confirmation interaction to be 
significant which was probably produced by the superior 
mean (4%) for the high RDF • no confirmation condition. 
There was a significant response by confirmation 
interaction reflecting the fact that confirmation had no 
effect with overt responding, but confirmation was 
inferior with covert responding. 

There was a significant difference in terms of 
difficulty of the content across the four units in 
this experiment. This type of finding is rather consistent 
in all of the following experiments as well. 
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Discussion 

The findings with respect to response demand 
frequency although not significant are consistent 
with those found during the first implementation 
with high RDF showing slightly better performance. 
With respect to the form of the response, again 
although not significant, shows the overt RD condition 
to be slightly superior particularly with high RDF. 

It does appear that if high RDF is employed it 
is probably best to demand overt responding, and that 
in this case the lack of confirmation may not make 
much of a difference* 
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VI. EXPERIMENT II 

Introduction 

This experiment was concerned with the effects 
of response demand frequency (RDF) and form of response 
demand (RD) on learning in the persistent medium of 
linear programed text. The hypothesis tested was 
that high RDF would be more effective than low RDF 
whether overt-selected, overt-underline, or covert 
RD was involved. 

Literature relating to effects of RDF was 
discussed in relation to Experiment I. It was 
pointed out that a number of studies using the 
persistent narrative text medium have demonstrated 
both general and specific attentional effects of 
overt responding. However, experiments using 
linear programed text are not so clear as to the 
effects of conditions of responding. As summarized 
by Anderson (1967), the large majority of studies 
show that students required to make overt constructed 
responses do not learn any more than students who 
"think" the answers, or read the program with the 
responses filled in. It should be noted that the 
comparison of constructed response and "thinking" 
represents a comparison of overt and covert RD 
conditions according to the Tosti-Ball (1969) model. 



It is not certain whether programs v/ith responses 
filled in should be considered high or low RDF 
conditions. It may be that a rhetorical question 
with the answer provided is nevertheless functionally 
equivalent to the same question with no answer. 
This might especially be expected to be the case 
with well -programed linear text with low error 
rates. Low RDF conditions are perhaps better 
represented by programs in which the questions are 
rephrased into direct statements. This form of 
low RDF condition is investigated in the present 
experiment. 

The present experiment compares overt-selected 
and covert RD conditions, together with an overt- 
underline condition which had not been investigated 
previously. The main purpose of the RD variable in 
this study is to provide an assessment of RDF-RD 
interactions. While no effects of RD conditions 
are expected by themselves, the effects of RDF may 
possibly be found to depend on the RD condition. 
Furthermore, certain types of students may do better 
using one RD form rather than another'. 
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Method 

Design and Analysis , The basic plan of tlie 
experimental design is shovm in Table 8. The students 
were randomly divided into six groups, each group 
receiving a particular sequence of 3 RD conditions in 
CPT 4, 6, and 10. Groups 1, 2, and 3 are trained 
using high RDF materials in all CPT units while groups 
4, 5, and 6 use low RDF materials. 



Table 8 
Plan of Experiment II 



Group 


Response 

Demand 

Frequency 


Cumulative Posttest Unit 




4 


6 


10 


1 


High 


Overt-selected 


Overt-underline 


Covert 


2 


High 


Overt-underline 


Covert 


Overt-selected 


3 


High 


Covert 


Ovort-selected 


Overt-underline 


4 


Low 


Overt-selected 


Overt-underline 


Covert 


5 


Low 


Overt-underline 


Covert 


Overt-selected 


6 


Low 


Covert 


Overt-selected 
— r 


Overt-underline 



It should be pointed out that the sequences 
for the first three groups are arranged to form a 
3x3 Latin square, and the sequences for the second 
three groups involved the same arrangement. Since 
the RDF conditions are assigned to independent groups, 
this variable is evaluated on a between-group basis* 



The use of two Latin squares, each with a particular 
level of RDF, permits the examination of the interaction 
with RDF of each within- student variable forming the 
Latin square. In particular, the RD and RDF x RD 
effects of interest in the present experiment are both 
within- student effects. Since effects of RDF are 
predicted, the evaluation of this effect with lesser 
precision as a between-student variable was regarded 
as conservative. 

< It may also be noted that the present design 

permits the examination of interactions between units 
and the variables of primary interest. Failure of 
the unit interaction to appear as significant would 
tend to support the generality of the effects of 
the primary variables acros? unit and content. In 
addition, suci: results would indicate that effects 
of the sequence of RD conditions were negligible. 

Materials and Procedures . The segments involved 
in this experiment were 4.1 - 4.3, 5.4 - 5.6, and 8.1 - 
8.3, and the medium used was linear text (see Table 2). 

Linear programed texts were developed by the 
RULEG and EGRUL methods of programing (Rule-example; 
example-rule). These are essentially programing 
methods of presenting a rule (definition, principle) 
and having the student identify an example of the rule 
(from 2, 3, or 4 choices), or presenting an example 
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and having the student identify the rule or principle 
which is depicted in the example (Markle, 1964).. 
Variations of the RULEG-EGRUL method which were used 
are EG -EG and RUL-RUL. 

It is important to note that although confirmation 
of responses is ordinarily an important part of 
programed instruction, confirmations were deleted in 
the first implementation in order to obtain valid 
data on student frame responses. In this run confirmation 
was provided in all cases except in Experiment I where its 
•effect was experimentally investigated. The first three 
modules of linear text were presented in standard 
format; i.e.- there was a question for every information 
frame. This defined the high response demand frequency 
(HRDF) dimension. The only difference in presentation 
among these modules was in the form of response required 
of the student. 

In module 1 (HRDF-written RD) , students were 
instructed to respond to each frame by writing their 
selection (A, B, C, or D) on the frame answer sheets. 
In module 2 (HRDF-underline RD) , students were instructed 
to respond to each frame by underlining their selection. 
In module 3 (HRDF-covert RD) , students were instructed 
to read each frame question and think the answers to 
themselves. 

Modules 4, 5, and 6 of the linear text covered the 
same material as the first three modules, but the 
frequency of response demand varied within the presentations. 
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Instead of asking a question for every frame, questions 
were asked for every second or third frame. In *'no- 
question" frames, examples or principles, which would 
be deleted when questions were deleted, were reworded 
in statement form; e.g., instead of asking, "Which of 
these situations best exemplified principle X?", the 
frame was followed by a statement, such as "An example 
of principle X is..." Module 4, 5, and 6 differed from 
each other in the form of response demand similar to 
modules 1, 2, and 3. 
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Results 

Mean student performance as a function of response 
demand frequency (RDF), and form of the response demand 
(RD) is given in Table 9. 

Table 9 

Mean Percent Correct Responses for Conditions of Instruction 



in Experiment II 





Response Demand Frequency 




Response Demand 


High^ 


Low 




Overt underline 


69.53 


70.51 


70.02 




Overt selected 


72.17 


66.39 


69.28 




Covert 


70.45 


72.65 


71.55 






70.72 


69.85 







The analysis of variance on this data (see Appendix A> 
shows no significant difference between the RDF conditions 
nor among the RD conditions. Although not significant 
there was a tendency for a RDF x RD interaction with overt 
selected better with High RDF (5.5%), covert slightly 
better with Low RDF (2%) and no difference for the overt 
underline condition. 

As was found in Experiment I there was a significant 
difference in difficulty level among units. 



Discussion 

In the first implementation there was a consistent 
superiority for High RDF that was not replicated in this 
implementation. With respect to RD the finding of no 
overall effect is not inconsistent with prior research. 
When the interaction effects are considered, the overall 
conclusions are similar to those made in the first imple- 
mentation as well as other studies. Performance was 
better when the response form required on the test was 
the same as that used in the instructional program^ regard- 
less of which form was used, ^ince the test required 
overt selected responses it is not surprising to find 
that training with overt-selected responses is somewhat 
superior particularly with high response demand frequency. 



VII. EXPERIMENT III 

Introduction 

This experiment investigated the effects of 
response demand frequency (RDF) and management 
frequency (MF) in two different media. Specifically, 
the hypothesis tested was that HRDF presentations 
with either high, medium, or low MF would be superior 
to a low RDF, low MF presentation. The Computer 
Assisted Instruction (CAI) and Audiotape- Intrinsically 
Programed Booklet (AT- IP) media selected for this 
experiment provided capability for the manipulation 
of MF, and enabled a test of the generality of the 
findings concerning RDF and MF. 

The results of the first implementation indicated 
thst a high level of performance was obtained 
where CAI was used. In order to assess whether this 
high level of performance was* due to the medium or the 
content itself, the CAI and Audiotape segments were 
also presented in a script version. 

It should be noted, hoxirever, that the primary 
purpose of Hypothesis III is not to compare different 
media. It is rather to assess the optimal conditions 
for learning within media. 



Method 

Design and Analysis . The basic plan of the 
experiment is presented in Table 10. The students 
were randomly divided into five groups, each group 
receiving one of four combinations of RDF and MF 
conditions of instruction in both CPT Units 5 and 13. 
The AT- IP media was employed for the presentations 
in Unit 5, and CAI for presentations in Unit 13. 
As table 10 indicates, a script version of each 
medium was. prepared as the fifth condition. This 
arrangement represents a "mixed" design, with RDF- 
MF conditions as the between-student variable, and 
unit-media combinations as the repeated measure 
variable. • 

Table 10 





Plan of Experiment 


III 




Group 


• , 

Cumulative Posttest Units 




3 (AT- IP) 


5 (CAI) 




1 


High RDF - High MF 


High RDF 


- High MF 


2 


. High RDF - Medium MF 


High RDF 


- Medium MF 


3 


High RDF - Low MF 


High RDF 


- Low MF 


4 


• Low RDF - Low MF 


Low RDF - 


Low MF 




(ATS- IP) 


(CAIS) 




5 


High RDF - High MF 


High RDF 


- High MF 



Since differences in performance between CPT 
Units 5 and 13 could be expected from differences 
in difficulty of content, test items, or quality 
of programing, in addition to differences between 
media, the present design does not provide a 
meaningful direct comparison between media. The 
design does provide an estimate of the effects of 
combinations of RDF and MF averaged over both media. 
In addition, the interaction between RDF-MF conditions 
and unit-media combinations could be examined to 
determine if the effects of the presentation variables 
were comparable in both media. 

The analysis of variance followed standard 
procedures for "mixed** design as modified for an 
unweighted means analysis (see Winer, p. 374). 

Materials and Procedures . The segments 
involved in this experiment were 4.4 - 4.7 and 12.1 • 
12.4. The medium used in segments 4.4 - 4.7 was 
audiotape with an intrinsically programed booklet, and 
the audiotape script with the booklet. Computer- 
Assisted Instruction (CAI) was used in segments 12.1 - 
12.4. A script version in intrinsic format was also 
developed for these four segments. 

As originated by Norman A. Crowder, the intrinsic 
programing technique consisted of routing a student 



through a "scrambled" text on the basis of his 
response. Each response directed him to a different 
page of the text; thus, the student could hot read 
through directly and sequentially. 

Combining the intrinsic programing technique 
with an audiotape was a WLC innovation. In this 
teaching mode, €he information was presented via 
the tape. While the student listened to it, he 
also looked at a summary page in the text which 
contained the precis of what he was hearing. He then 
stopped the tape and followed the instructions at 
the bottom of the summary page, directing him to a 
page containing a question which tested the 
information given on the tape and summary. Each 
response to the test item referred the student to 
another page which informed him of the accuracy of 
that response. Thus, the student would select the 
alternative which he thought was correct, turn to 
the page indicated for that alternative, and find 
out if he had made a correct selection. If he had 
selected the correct response, he was instructed 
to go on to another summary page, which he read while 
listening to the next audio portion. If his response 
was partially correct or incorrect, he was either 
told the nature of his error and instructed to proceed 



as described above, or he was instructed to return 
to the summary or question page to study the 
information again and select another alternative. 
This process of interaction between tape and text 
continued throughout the segment. 

The tape, which contained the content of the 
segment, remained the same throughout the four 
modules. The text differed as follows: In module 
1 a question was asked for each informational frame 
(HRDF) . Based on his response, the student was 
always branched to a page where his answer was 
discussed and confirmed or rejected (high management 
frequency). In module 2 a question was asked for 
each frame (HRDF) , but the student was branched on 
the basis of his response for only 501 of his responses 
(medium management frequency) . In module 3 a question 
was asked, for each frame (HRDF), but the student 
was never given feedback nor confirmed; i.e., he was 
never branched as a result of his selection. He 
simply went on to the next question (low management 
frequency). In module 4 only three or four questions 
were asked throughout the programed sequence (LRDF) , 
and the student was not branched on the basis of his 
responses to the three or four questions (low management 
frequency). Module 5 employed the same text as -".-^lule 
1 but the script was used in place of the tape. 
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All of the three components of the CAI 1500 
system (CRT, audio, and image projector) were utilized 
in the implementation of the four modules. The first 
four modules exactly paralleled the first four modules 
used in. the audiotape-IPB segments. ' Module 5 was 
similar in that a script version of the High RDF - High 
MF condition was developed. The following pattern 
was followed in developing the CAI materials. The 
informational frames were presented on the CRT screen 
and image projector. The questions, which were often 
situations in which the student had to decide the best 
course of action, were presented: 1) on the audio, 
where the situation was described; 2) on the image 
projector, where pictures of the situation were 
presented along with the audio; 3) on the CRT screen, 
where the student was asked to select an answer from 
3 to 5 choices. The student's selection, accompanied 
by feedback, was displayed on either the CRT screen or 
the audiotape, and occasionally on the image projector. 
This feedback consisted of the reason(s) why the 
selected answer was correct or incorrect. 



ERIC 
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Results 

Mean student performance as a function of response 
demand frequency (RDF) , and management frequency (MF) 
conditions is given in Table 11. 

Table 11 

Mean Percent Correct Responses for Conditions of 



Instruction in Experiment III 



Conditions 


Mean 


(Audiotape/CAI Versions) 




High RDF - High MF 


72.15 


High RDF - Medium MF 


68.98 


High RDF - Low MF 


68.18 


Low RDF - Low MF 


69.26 . 


(Script Versions) 




High RDF - High MF 


73.02 



As can be seen in Table- 11, the high RDF - high MF 
conditions whether script or tape/CAI are superior to 



the other conditions. The analysis of variance (see 
Appendix A ) however, showed no significant differences 
among the five conditions. As usual, there was a 
significant difference among the unit-media combinations. 
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Discussion 

In the last implementation the findings in this 
experiment were somewhat counter in. dtive in that 
the high RDF-high MF condition was expected to be 
equal to or better than the other conditions; however, 
it was actually worse in most cases. It was noted 
that where management was not provided, no confirmation 
was given, however, confirmation was always in the high-high 
condition. Confirmation was added in cases where it 
was missing in order to control any possible mediating 
effects. The findings in this implementation appear 
to be more consistent, although no significant 
differences were found among the five conditions in 
the analysis of variance (see Appendix A ). It is 
interesting to note the lack of a significant 
difference between the script versions and the hardware 
bound versions, particularly considering the differences 
in the costs of development and production. 



VIII. EXPERIMENT IV 

Introduction 

One major goal of the preceding three hypotheses 
was to demonstrate that students will generally perform 
better on presentations with high RDF. Numerous 
exceptions to this principle, however, have been a 
source of many criticisms recently leveled at linear 
programed instruction, which has high RDF as an 
elemental, identifying characteristic. 

Although PI has been "the only media group which 
has made some effort to back up its claims with sound 
data" (Tosti and Ball, 1969), and although a number 
of studies indicate that "small -step" programs are 
generally more effective than low RDF presentations, 
, there has been an embarrassing number of cases in 
which students do not like, and do not perform well 
on, linear programed instruction. 

One of the chief complaints has been that some 
students are able to comprehend relatively large units 
of content material, and are thus bored by frame 
sequences in which responses to small pieces of 
information are required at frequent intervals. If 
it is' true that individual students vary in their 
ability and preference for differing RDF presentations, 
then criticism of PI for failure to consider individual 
differences along the RDF dimension is well taken. . 
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Csanyi (196S, 1961) has suggested a procedure 
by which this apparent RDF limitation might be 
effectively dealt with. He has extended the format 
on the basic linear program so that students who 
do not require a frame sequence of instruction do not 
have to go through it. In this "syndactic text" 
program, the student first receives, and is tested 
on, a relatively large unit of information presented 
in textual form. If the student passes the test on 
the sumnary statement, he proceeds directly to the 
next summary statement. However, if the student does 
not pass this test, he is immediately branched to a 
frame sequence covering the same material. Csanyi 
has reported that this technique produces su'perior 
performance in many different types of students. 

Experiment IV is a direct test of the effective- 
ness of the syndactic-text pr.ocedure. It is hypothesized 
that students unable to achieve satisfactory levels of 
performance after practice on a low RDF presentation, 
will show superior levels of performance if remediated 
on a high RDF presentation, as opposed to an additional 
low RDF presentation or no remediation. If results 
support this hypothesis, one means of adapting the 
Tosti-Ball (1969) model to individual differences in 
instructional needs is suggested which may have 



considerable generality. For example, a low RDF 
audiotape presentation may be found ineffective 
for some portion of the students, who might then 
be remediated on high RDF linear text. The entire 
instructional sequence might prove more efficient 
than either audiotape or linear text would be if 
used alone. 
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Method 

Design and Analysis. The plan of this experiment 
is shovm in Table 12. The students wore randomly divided 
into six groups, each group receiving a different 
sequence of three experimental methods in CPT Units 
3, 8, and 12. In all conditions, the students received 
the same syndactic-text summaries; the indicated 
methods of instruction refer to the type of 
supplementary material used when an inadequate level 
of performance was achieved on the quiz followij^g a 
summary. 



Table 12 
Plan, of Experiment IV 



Group 


Cumulative Posttest Unit 




2 


8 


12 


1 


High RDF 


Low RDF 


None 


2 


Low RDF 


None 

* 


High RDF 


3 

• 


None 


High RDF 


Low RDF 


4 


High RDF 


None 


Low RDF 


S 


Low RDF" 


High RDF 


None 


6 


None 


Low RDF 


High RDF 



•It may be noted that all six posMble sequences 
of three conditions are used in this design. The 
sequences are arranged to form two different 3X3 



Latin Squares, one square invclving Groups 1, 2, 
and 3, and the second square involving Groups 4v S> 
and 6* The difference between these squares may be 
seen in the reversed sequence of conditions in 
Units 8 and 12. The consequence of this reversal of 
sequence is that different components of the Method 
X Unit interaction are confounded with group differences 
in each square. Since the groups are randomized, 
significant differences among groups may legitimately ' 
be interpreted as arising from the confounded Method 
X Unit interactons. However, the remaining components 
of this interaction are independently estimable from 
the residual variation in each square, after removal 
of Unit, Method, Group variation. Thus a second, and 
more precise, test of the Method X Unit interaction is 
obtained on a wi thin-student basis. 

The present design also permits the examination 
of Square X Unit and Square X Method interactions, 
which in this case must be interpreted as arising 
from effects of the sequence of methods. 

The analyses were zonducted according to the 
procedure described by Winer (1962, p. 549) as 
modified for an unweighted means analysis. 
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Materials and Procedures . The segments involved 
in this experiment were 2.6 - 2.8, G.l - 6.3, and 
11.1 - 11.3, and the medium used was syndactic text 
(see Table 2). 

A syndactic text is essentially a series of 
linear programed frames each preceded by a brief but 
complete summary of the information presented in the 
frames. Students work through the syndactic text by 
reading the first summary statement and taking a 
summary quiz of 5 to 8 questions. If the student 

« 

answers all summary quiz questions correctly, he reads 
the second summary, takes summary quiz 2, etc. 

The student who incorrectly answers one or more 
questions of a summary quiz is required to study the 
linear programed sequence which is identical to the 
linear text. It was developev! by the RULEG or EGRUL 
method of presenting small bits of information 
accompanied by examples of the concepts being taught. 
At the end of the programed sequence ^ the student 
retakes the summary quiz* Regardless of his performance 
when he retakes the summary quiz^ he goes on to the 
next summary statement and repeats the procedure* Non- 
research segments of syndactic te,^t were implemented 
according to the procedure given above* 

In the iirst implementation approximately 70% 
of the student£> scored 100% on the summary quizzes 
-and therefore, the effects of the two programed sequences 

M 
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(HRDF vs LRDF) could not be assessed. In this run 

the students proceeded through the sequences regardless 

of their quiz score. 

The first module of these research segments of 
syndactic texts was presented in the same manner as 
non-research segments, except that the students 
proceeded through the linear sequence (HRDF) regardless 
of his quiz score* 

The second module of syndactic text was identical 
to module 1 except that it was characterized by a low 
response demand (LRDF) program. Instead of having RULEG 
question frames, examples were simply ^iven in statement 
form. This sequence of statements was referred to as a 
"detailed summary statement." There were no questions 
asked in the detailed summary statement. 

The third module in these nine segments was 
represented by summary statements alone (no remediation, 
no RDF). The student read a summary statement, took 
the summary quiz, re-read the summary and then proceeded 
to the next summary statement regardless of his score 
on the summary quiz. 
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Results 

Mean student performance as a function of- the 
three remediation conditions is given in Table 13. 

Table 13 

Mean Percent Correct Responses for Remediation Conditions 
in Experiment IV 





Remediation 


Condition 






Linear 


Detailed 


Re-read 




Sequence 


Sequence 


Summary 




(High RDF) 


(No RDF) 


CPT Score 


69.70 


74.39 


70.97 



Although there is a 4% difference in favor of 
the detailed summary as opposed to the linear 
sequence and a repeat of the summary^ the analysis 
of variance (see Appendix A ) . indicates a significant 
difference only for units which is expected. 



Discussion 

As was found in the first implementation a high 
percentage of students (approximately 70%) did not 
(according to their summary quiz performance) ne^ed ' 
the remediation. It would appear that the syndactic 
text summaries are rather effective. Indeed, it is 
the case that the students reported a high degree of 
approval for this type of instruction. It would 
appear that any of the three forms of remediation 
would be equally effective. 



IX. EXPERIMENT V 



Introduction 

This experiment investigated the effects of 
advance organizers as used within the medium of syndact 
text. Although Programed Instruction (PI) has been 
"the only media group which has made some effort to 
back up its claims with sound data" (Tosti and Ball, 
1969) , and although a number of studies indicate 
that "small -step" programs are generally more 
effective than low RDF presentations, there has been 
an embarrassing number of cases in which students 
do not like, and do not perform well on, linear 
programed instruction. 

One of the chief complaints has been that some 
students are able to comprehend relatively large 
units of content material, and are thu3 bored by 
frame sequences in which responses to small pieces of 
information are required at frequent intervals. In 
going through the small steps, a student may lose 
track of where the specific bit of information fits 
in the overall unit of instruction. Ausubel (1965) 
has stated that the advantage of deliberately 
constructing special organizers for each new unit 
of instruction is that the learner can enjoy the 
advantages of a subsumer in \^hich the ne\>r material 



can be related to stable elements already in his 
cognitive structure which both (a) gives him a 
general overview of the more detailed material in 
advance of his actual confrontation with it, and 
(b) also provides organizing elements that are 
inclusive of and take into account most relevantly 
and efficiently the particular content contained 
in the material* Any existing subsumer in the 
learner's cognitive s::ructure, which he could 
independently employ for this purpose, self- 
evidently lacks particularized relevance and 
inclusiveness for the new material and would hardly 
be available in advance of initial contact with it. 
Although students might be able to improvise a 
suitable subsumer for future learning efforts after 
they become familiar with the material, it is 
unlikely that they would be able to do so as 
efficiently as a person sophisticated in both 
subject matter content and pedagogy. A "content 
map" showing the relationships of all topics in a 
graphic display /as used as the format for the 
advance organizer. 



Method 

Design and. Analysis . The plan of Experiment 
V is given in Table 14. 

Table 14 
Plan of Experiment V 





Cumulative Posttest Unit x 


Group 


14 


11 


1 


Content map 


Content map 


2 


No content map 


No content map 



The students were randomly assigned to one of the 
two groups resulting in a repeated measures analysis 
of variance desig.i. 

Materials and Procedures . The segments 
involved in this experiment were 5.1 - 5.3 and 8.4 - 
8.6 and the medium used was syndactic text. 

A syndactic text is essentially a series of 
linear programed frames each preceded by a brief 
but complete summary of the information presented 
in the frames. Students worked through the syndactic 
text by reading the first suirmary statement and 
taking a summary quiz of S^to 8 questions. If the 
student answered all summary quiz questions correctly, 
he read the second summary, took summary quiz 2, etc* 
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The student who incorrectly answered one or more 
questions of a summary quiz was required to stuily the 
linear programed sequence which was identical to the linear 
text discussed previously. It was developed by the 
RULEG or EGRUL method of presenting small bits of 
information accompanied by examples of the concepts 
being taught. At the end of the programed sequence, 
the student retook the summary quiz. Regardless 
of his performance when he retook the summary quiz, 
he went on to the next summary statement and repeated 
the procedure. 

Figure 2 provides a sample of the content maps 

that were used as advance organizers. At the beginning 

of each of the six experiments in this experiment, 

the students in group one were given the following 

instructions in regard to the use of the content maps. 

Before reading the text^ study the enclosed 
"content map.^^ This chart outlines the 
si*'>ject matter taught in this segment. It 
vindicates both the order in which the various 
tbpits are disciissed and the intended 
relationships among the various topics. 
Read the content map starting from the 
bottom. The number in the bottom right 
corner of each box indicates the order 
in which each item is taught. Parallel 
paths indicate relationships among ' 
slightly diverse topics. The number 
in the upper left corner of certain boxes 
indicates that these are higher order 
topics which incorporate the topics 
discussed in the boxes below it. These 
are usually major points in each of the 
summaries. 



studying the content map before reading 
the text will help you to recognize 
clearly the relationships among the " 
ideas taught. You should find it 
helpful to refer back to the content 
map as you study each of the summaxies. 



Figure 2. 

Sample 
Content 
Map 
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Results 

Mean student performance as a function of 
content map usage is given in Table 15 • 

Table 15 



Mean Percent Correct Responses 
for Conditions of Instruction in Experiment V 



Group 


Mean Percent Correct 


Content Maps 


74.41 


No Content Maps 


77.35 



It can be noted that there is a '3% difference 
in favor of no content maps. However, the analysis 
or variance (see Appendix A ) showed no significant 
difference with respect to the maps. 

There was no significant unit effect nor any 
interaction between units and map'^. 



Discussion 

The finding of no significant difference for 
the effect of maps, it should be pointed out, is 
not an indictment against advance organizers, but 
it may indicate that if advance organizers are 
generally effective, the content maps as used here 
were not. However, this finding may relate to the 
fact, as Ausubel (1965) has reported, that the 
facilitating effect of organizers occurs more 
readily for students with relatively poor verbal 
ability who tend to structure material less 
effectively. Considering the relatively high 
ability level of the general USNA population 
it is perhaps not surprising to find ho significant 
differences. 

This finding is consistent with the results 
of the subsidiary analy^.s relating to the use of 
maps as reported in Chapter X. 



X. SUBSIDIARY ANALYSES 

Introduction 

The cumulative posttest unit., were the , 
fundamental units of instruction for the research 
conducted in Experiments I through V, Performance 
on the cumulative posttests (CPT's) was the dependent 
measure used in each of the experiments. The CPT's 
were norm-referenced rather than criterion referenced 
tests. In the previous run of the course, the 
statistical analyses in each of the experiments 
were conducted using performance on the criterion- 
referenced progress checks as the dependent measure 
in addition to the CPT's. No great drlsimilarities 
were found in the results using these two different 
measures, although the CPT results did perhaps provide 
better precision. 

The following subsidiary analyses were developed 
shortly before the course was implemented the second 
time (fall 1970) and the progress checks alone were 
used as the dependent measure. Basically the three 
subsidiary questions to be iiscussed in this chapter 
were replications of variables investigated in the 
previous experiments. The variables manipulated were 
the use of the (1) content maps, (2) script versions 
of various audiotape segments and (3) special answer 
forms for the syndactic text summary quizzes. 
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Subsidiary Analysis I 

The use of the content maps and special answer 
forms was investigated in segments 2.1, 2.9, 3.5 and 
6.5. The medium used was syndactic text as discussed 
in Chapter. IX. The content maps used as advance 
organizers was also discussed in Chapter IX. The 
special aiiswer sheets were used by the students in 
answering the summary quizzes in these segments. The 
student used a special crayon in marking his answer 
on the sheet and in a few seconds after marking his ^ 
answer, the correct answer would appear. The question 
of using this type of revealed answer form arose when 
it was discovered after the first run that very few 
students had to go through the programed sequences. 
Going through the programed sequences naturally added 
considerable time compared to just reading the 
summarier, and it was felt that with the correct 
answers available to the student in the text, the 
temptation^ to ensure that going through the remedial 
sequences was not necessary^ was very real. If the 
student*? could achieve an acceptable level of the 
progresi, checks it really didn^t matter whether they 
went through the linear sequences, re-read the 
summary or just went on to the next summary. The 
point was however, that it would be difficult to 
determine which procedure would be most effective if 
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it could not be positively stated that the correct 
procedures were followed. 

The basic design of this analysis is shown 
in Table 16. The students were randomly assigned 
to groups. 



Table 16 
Plan for Subsidiary Question I 



Group 


Segment 


2.1 


2.9 


3.5 


6.4 


1 


Map -RAF * 


No Map -RAF 


Map -No RAF 


No Map -No RAF 


2 


No Map -RAF 


No Map -No RAF 


Map -RAF 


Map -No RAF 


3 


Map -No RAP 


Map -RAF 


No Map -No RAF 


No Map -RAF 


4 


No map -No RAF 


Map -No RAF 


No map -RAF 


Map -RAF 



RAF « Revealed answer form 



r>:- --';i, •---•---:^-«.• 
K^-C-^•.-..-^"■■^ 1 ■ 
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Results. The mean student performance £br the four 
conditions is given in Table 17. 

Table 17 

Mean Percent Correct oh Progress Checks 







the Conditions of 


Instruction in 






Question I 








.i^sWe^^^ Shee t 




■ No Ahsw^^ Sheel: ; 








fl.46 ■ 




90; 85 






^ ' 1 . No Map 


88.31 




89>61 


88.96 






89V88 


90.23 





The analysis of variance of this data (see 
Appendix B) indicates no significant differences with 
respect to the use of the maps or the special answer 
sheets. As has been seen with most of the analyses ^ 



a significant difference was found across the segments, 
but there was no interaction between segments and 
either of the two variables. 

the finding of no differences with the maps is 
consistent with that found in Experiment V where the 
dependent measure was the norm-referenced cumulative 
posttes.^ A few of the students report that the 
special answer~^heetS'did tend- to remove the temptation 
of looking at the quiz answers before responding, but 
the forms were so unreliable that the students became 
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irritated atltheiT use. The correct answer sometimes 

took several 'minutes to ap^^^ In addition, the 
additional cost of thiese forms did not warrant their 
continued uise^ The fact was that the students 

learned equal# well whether contrplletf by the special 
answer sheet or not. 



mc 
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Subr tdi af y Aiiaiys is II 

Ihe use of the content maps as advance 
organizers was investigated again in segment 7.5 
Which was originally programed in the medium of 
audiotape'/panel book, the use of the. tape script 
in place of the audiotape Was; also investigated 
in this segment, the plan of this analysis is 
ihowi in Table 18. The students wefe randomly 
assigned to a group. 
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Table 18 
for 



Question II 



Segment 



Group 




1- 


Audiotape/PB-Map 


2 


Audiotape/PB-No Map 


S 


i 

Audiotape Script/PB-Map 


4 


Audiotape Script/PB-No xMap 



:ERlC 



Results . The mean student performance for the four 
. conditions is giv^ri in Table 19. 

table 19 " 
Mean Percent Correct oh Progress Checks for the Conditions 

> ■ - ■ - '. " " : ■ 

of liist ruction in Question il 





Map 


; No Map 




itodiptapie, : 






- 83i32 ' 


Tape-ScMpt i 




82. 00 


. 81; do 




84.5d 


; 82.i2 





the analysis of variance of this data (see 
Appendix B ) showed no significant differences due 
to the me4ia (script ys tape), the use of the content 
map, nor was there a significant interaction between 
these variables. 

^ Again, the findings in relation to the maps was 

i 

consistent with the previous analyses. The finding 
of a lack of difference between the use of the 
scripts of the tapes in conjunction with the panel 
book as opposed to the tapes themselves is an important 
finding. ]> gives the instructional system and student 
an option in tl\e us« of th^ materials without apparently 
jeopardizing his pei^ormanee. This is consistent with 
the lack of difference found in Experiment HI between 
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scripts and, the hax^are bouiid (audiotapes and GAI) 
versiohs of 'ehe niateri.als..' A similar finding will 
be iidted. in the last iubsidiary analysis on the . 
fbtipwing pages. , 




Subsidiary Analysis ill. . 

the use bf the contefit maps and the script 
Version q£ the aiidibtape w^ith panel book and the 
intrinsiGaily programed bopKlet was experimentally 
manipulated across segments ^9^^^ 9.2, 10. 1 and 
10.2. The basic design of this analysis is shown 
in table 20. .The students were randomly assigned 



9S 



to the 


groups. 


ry Question iM 




[ Segment » 


' Group 


9.1 


9.2 


1 


Audiotape Script/EB 
Map 


Audiotape/PB^ 
Map ^ 


2 


Audiotape/PB 
• Map 


Audiotape Script/PB 
Map 


3 


Audiotape Script/PB 
No Map 


Audiotape/PB 
No Map 




Audiotape/PB 
N6 Map 


Audiotape Scripc/PB 
No Map 




Segment 




Group 


10.1 • ^ 


10.2 


1 


Audiotape/ IP 
No Map 


Audiotape Script/IP 
No Map 


2 


Audiotape Script/IP 
No Map \. 


Audiotape/IP 
No Map 


3 


Audiotape/ IP^^t 
Map 


Audiotape Script/IP 
Map 


4 


Audiotape Script/IP 
Map 


Audiotape/IP 
Map 



i 



Results . The mean stuaent performance for the 
conditions of instruction in Question III is given in 
Table 21. \ > , " 

; Table 21 

Mean Percent Correct on Progress Checks for the Conditions 
of Instruction in Question III 























Segiheht 
















94 




9>.i ; 


10.1 




10.2 






Script 


Tape ; 


Script : 


Tape 


Script 


tape 


Script 


Map 


:92.22 


94.55 


92,>3' 


92*22 


95.00 


.94.55 


92.73 


89.00 


No Map 


89.09 


91.00 


88.00 


91.82 


97.27 


98.89 


94.44 


89.09 



The analysis of variance of this data revealed no 
significant differences between media (script vs tape) 
across Part 9 nor across Part 10 (see Appendix B) . 
There was no significant main effect for the use of the 
maps across the four segments (x for maps =92.88, 
X for no maps =-92.45). /s usual there was a significant 
difference between the two parts with Part 9 slightly- 
more difficult than Part 10, and a significant difference 
between the ^wo segments of Part 10 with 10.2 somewhat 
more difficul^. There was also a small segment by 
media interacti^ in Part 10 where audiotape was somewhat 
superior in segment 10.2 but slightly reversed in 
segment 10.1. 
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Agitih: it ca^: be noted that these fihaings are 
consistent with aH^ the- previous analyses where 
the use of. content maps and scripts were manipulated, 
.Although there were hp sigriificaht differences, the 
'consistency of the many analyses' using both nqrm- 
and criterion measures give some weight 

to the. previous cqhciusipns . 



- 1 
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XI. SUMMARY OF SPRING AND FALL RESULTS 

this research effort was directed at answering 
the central question of whether variations of conditions 
of instruction in the presentation design domains as 
posited by Tbsti and Ball (1969) are of greater 
importance than variations 4n the media, domain. Tosti 
arid; Ba/il' s position has riot received strong 
Experimental support from this series of studies . 
i^deedj iri -Experiiiierit I of the spring run where the 
preseritatiori design Was held constarit, a significant 
difference was found between media. Although this 
does not necessarily refute the basic assumptions of 
the Tosti and Ball model, it does indicate that other 
variables relating to the production of instruction 
particularly via transient media (videotapes arid 
audiotapes) should.be considered. On the other hand, 
one can see many cases whcveVq significant differences 
were found when the medium was constant and the 
presentation design was varied. 

In investigating the question of whether 
substantial effects of student achievement could be 
produced by manipulation of presentation variables 
over large segments of instruction. Experiment 11 
of the spring run provided the most conclusive 
evidence. It was found that with linear text, the 
high response demand frequency condition was 



consistently superior to the low response demand 
frequency condition. A facili.tative effect does 
appear to be produced by the insertion of a 
significant number of questions in the instructional 
material.' Further investigations need to be conducted 
with respect to this variable as implemented with 
transient media (see Experiment I spring and fall 
runs). 

Although major differences were hot found with 
respect to. the form of response required of the student, 
the trend in Experiment II of the spring run was that 
the overt- selected response condition was slightly 
better than either overt-spoken or covert. This 
finding is generally in concert with the major body 
of prior research that indicates that the form of 
response utilized within the instructional materials 
should be similar to that required of the student 
on the tests of achievement on those materials. 
In Experiments I and II of the fall implementation, 
no significant differences were found with respect 
to the main effect of the form of the response 
demand (RD). However, if one has a high 4lDF 
presentation it was felt that the most, effective 
RD might be overt- selected which is in concert with 
the spring findings. 



Perhaps the least conclusive finding related 
to. manipulations of management frequency (MF) in 
conjunction with variations . in response demand 
frequency (RDF) as indicated in Experiment III 
of the ispring run. It appeared that the most 
^ efficient condition was the lise of moderate levels 
of management frequency (M?) in conjunction with 
high response demand frequency, (RDF) . However, an 
intervening ytoable ma^y^^^^^^^ been the fact that 
knowledge of results was hot ptovided in those 
cases where management was not manipulated. This 
may have produced variations in the experimental 
conditions that were not due to the major variables 
investigated. In the fall run, with confirmation 
added, the results showed no significant differences 
among the combinations of RDF and MF nor between a 
script version of the high RDF-high MF condition and 
the automated (tape or CAI) version. 

In Experiment IV of both the spring and fall 
runs the type of remediation provided after the 
syndactic text sumiparies had no effect on the students 
performance. They performed equally well. Indeed 
it was the case that the students generally performed 
so well on the syndactic text summaries that a precise 
test of the remediation conditions in this medium 
was difficult to obtain. 



Indetid the preponderance of findings Were 
"no significant differences"! In Experiment V 
(fall run) no significant differences were found 
with respect to the use uf "content maps" as 
advance, organizers. This was replicated in the 
subsidiary analyses of the fall run using progress 
checks as the dependent ^measure. In these same 
subsidiavy analyses performance^was almost identical 
where scripts of tapes were compared with the' use 
of the tapes themselves. With respect to both 
media and presentation variables, there were no 
consistently significant differences with respect 
to the type of learning or task required of the 
studeiit as seen in the first implementation. That 
is, the findings with relation to norm-referenced 
CPT performance were generally consistent with -^he 
findings in relation to progress check performance. 

Comparison among media with the same kind of 
control and precision devoted to comparison of 
presentation variables in the present studies may 
well have demonstrated numberous reliable differences 
among media, as was the finding in Experiment I 
of the spring run. However, these results do 
support the general conclusion that differences 
among media, if they exist, are no more substantial 
nor important than variation in difficulty among 
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units in the same media, resulting from variation 
in content programing skill, or test items. Thus 
it would seem that the problem of media selection 
should deserve no more attention from the 
educational technologist than he is willing to 
devote to these other sources of variation in 
student achievement. 

General Conciusions . With such a large 
number of analyses showing no significant differences 
one has the choice of accepting the fact that there 
is a high probability that no real differences exist 
or that the experimental designs and analyses are in 
someway at fault. 

Much of the prior research on media and various • 
presentation variables involved instructional materials 
of a short-term nature which usually covered one main 
content area, and were frequently conducted in an 
experimental environment. It should be noted that 
this research effort involved investigations over a 
semester's worth of instruction in a required course 
for a significant portion of the midshipmen at the 
U. S. Naval Academy. The content of the instructional 
materials covered topics in many disciplines (see 
Table 2). In addition, many of the experiments were 
replicated in a second implementation of the course. 
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The experimental designs were sound and 
employed tight control to ensure that the research 
data could be gathered without hindering the learner's 
opportunity to learn. The part and segment results 
indicated significant differences as small as 2.5 to 
3.0 percent. Although they were statistically 
significant they obviously were not educationally 
significant. Therefore, it was felt that it was 
unlikely that the lack of findings of effects of the 
primary experimental variables could be attributed to 
a lack of precision in the experimental designs and 
analyses. 

The above discussion does not mean however, that 
it has been established as fact that no real differences 
exist with respect to media and presentation variables. 
-What it does indicate is that with general prescriptions 
to the programers (i.e., for high RDF write a question 
for every fiame; for medium MF provide response 
sensitive feedback every other frame, etc.) and precise 
guidelines, based on a sytematic approach, for the 
revision of the instructional materials, almost any 
medium with almost any presentation design characteristics 
can be programed so that the learners can reach a pre-set 
level of acceptable performance. However, certain subject 
matter programed in some media, with certain presentation 



characteristics, will demand more effort and cost 
to show the desired level of performance. Some* may 
consider cost the most critical factor and opt for 
slightly less effective student performance. 
Others may well consider effective -student learning 
to be most critical, at whatever the cost. 

The topics for future research that would appear 
to have high payoff for instructional technology 
woiild be the investigation of optimum revision 
procedures and the delineation^ of the variables 
important in developing effective programing skill. 
Indeed, the programer needs to be an independent 
variable in future media research just as the 
teacher and test administrator have been recognized 
as independent variables in othisr research. Even 
when general prescriptions are given to a programer, 
be it in the development of a film, a tape, or 
printed material <^ - ^leat deal of latitude remains 
in his hands as to the appearance of the final 
product in its most minute detail. 



105 



REFERENCES 



Anderson, R. C. Educational psychology. In P.R. 

Farnsworth (Ed.), Annual Review of Psychology . Palo 

Alto, California: Annu a r Reviews, Inc., 1967. pp. 129-164. 

Ausubel, David P., and Anderson, Richard C. Readings in 
the Psycho logy of Cognition . New York: Holt, Rinehart 
and Winston", Inc., 1965, pp. 103-115. 

Bloom, B. S.-, Englehart, M. P., Furst, E. J., Hill, W. H. , 
^.Krathwohl, D. R. Taxonomyjof educational objettiv es: 
The Qlas s i f j cat i on^oi: . edu ca 1 1 onaK go alsT (Handbook I": 
Cbghiilivl Domai^) New Ydfk: Lbrigmafts, Green arid Co. . 
1956. 

C'de Baca, J., and Chadwick, C. B. PROMOD: A new 
approach to multi-level programming. NSPI Journal. 
1968, 7 (2), pp. 8-10. 

Crowder, N. A. Automatic tutoring by means of intrinsic 
programing. In E. H. Galanter (Ed.), Automatic teaching : 
The state of the art . New York, Wiley, 1959, pp. 109-116. 

Csanyi, A. P. An investigation of visual versus auditory 
programming in teaching foreign language pronunciation. In 
Investigations of the Characteristics of Programmed 
Learning Sequences . Pittsburgh: University of Pittsburgh, 
Programmed Learning Laboratory^ 1961. pp. 135-150. 

Csanyi, A. P. Analytical techniques for programed 
instruction 7.n foreign languages. Paper presented at 
the Annual Convention of the National Society for 
Programmed Instruction. May, 1965. 

Evans, J. L., Homme, L. E., § Glaser, R. The RULEG 
system for the construction of programed verbal 
learning sequences. Journal of Education Research, 
1962,. .55, pp. 513-518. 

flanagan, "j . C. Functional . education for the 70 's. 
Phi Delta Kappan, 49 (Sept. 1967), pp. 27-32. 

Gagne, R. M. The conditions of learning . New York: 
Holt, Rinehart and Winston, 1965. 



106 



Mager, Robert 'F. Preparing Instructional Object ives. 
Fearon Publishers, Palo Alto, California. 1962: — ~ 

Markle, Susan. Good Frames and Bad . New York: John 
Wiley and Sons, Inc., 1964. 

Tosti, D. T., and Ball, J. R. A Behavioral Approach 
to Instructional Design and Media Selection . 
Albuquerque, .1969. Behavior Systems Division, 
Westinghouse Learning Corporation. 

^ • . * 

Winer,. B. J Statistical Principles in Experimental Design, 
New York: McGraw- Hil 1, ; 19 6 2 . ■ 



ERIC 



APPENDIX A 
Experiment I 
Analysis of Variance 
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Source 

Response (R) 
Group (G/R) 
Student (S/GR) ' 

Units ..(U) 
Frequency (F) 
Cbrifirmsition- fC) 
FG 

RF 
RC 

Residual 
S(FMU)/GR 



< .05 
p < .01 



d£ 

\ 


MS 


F 


1 


20.430 


<1 


2 


17.938 


<1 


37 


27.666 




3 


389.832 


43.11** 


1. 


21 i 9 80 


2.43 


1 


23^.965 


2.43 


1 


18.551 


2.05 


1 


0.495 


<1 


1 


37.854 


4.19* 


4 


2.335 . 


" <1 


110 


9.044 





Experiment II <■ 
Analysis of Variance 



Frequency (F) 
Rows (R) 
FR 

Students, (S/FR) 

Units (U) 

Method (M) 

FU 

FM 

UM 

FUM . 

S X (UM)/FR 



1 


1.919 


<1 


2 


43.471 


2.87 


2 - 


8.132 


<1 


35 


15.695 




2 


84.763 


12.05* 


f 2. 


4.580 


<1 


2 


3.009 


<1 


2 


15.789 


2.25 


2 


1.377 • 


<1 


2 


1.402 


<1 


67 


7.033 





< .01 



108 



APPENDIX A 
Experiment III 
Analysis of Variance 



Source 

Methods (M) 
Students (S/M) 

Units (U) 
MU 

SU/M 

*p < .01 



d£ 


MS 




4 


10.737 


<i 


36 


14.417 




i 


176.679 


32.84* 


4 


1.528 


<1 


35 


5.380 





Experiment IV 
Analysis of Variance 



Square (Q) 
Groups (G/Q) 
Students (S/GQ) 

Units (U) 

Method (M) 

QU - 

QM 

UM/Q 

SUM/GQ 



1 


19.107 


<1 


4 


. 12.437 


<1 


35 


20.326 




2 


92.752 


13.08* 


2 


20.535 


2.89 


2 


.675 


<1 


2 


10.124 


1.43 


4 


2.697 


<1 


68 


7.089 





'p < .01 



Experiment V 
Analysis of Variance 



Content Map (M) 


1 


14.854 


Students (S/M) 


39 


6.476 


Unit (U) 


1 


.054 


U X M 


1 


2.784 


U X S/M 


39 


5.551 



1, ■ Ot»i». 
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APPENDIX B 
Subsidiary Question I 
Analysis of Variance 
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Source 

Sequences 
Students 
Segments 
Sheet (S) 
Map (M) 
S X M 

Students X Seq 
*p < .01 



df 

3 
37 
3 
1 
1 
1 

111 



MS 

24.18 
95.46 

363.89 
4.84 

193.09 
36.55 
67.99 



F 

<1 

5.35* 
<1 

2.84 
<1 



Subsidiary Question II 
Analysis of Variance 



Media (M) 
Content Map (C) 
M X C 

Students /MC 



1 


4.73 


< 1 


1 


28.88 


< 1 


1 


55.08 


<1 


37 


125.68 
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APPENDIX B, 
Subsidiary Question III 
Analysis of Variance 



Source 

Sequence (Q) 

CP 

QCP 

Students 

Part (P) 
Content Map (C) 

Within Part 9 
Segments (So) 
Media (Mq) 
So X Mq 
C§9 

CMg 

Within Part 10 
Segments (Sin) 
Media (Mjo)^" 
ho X Mio 
CSio 
CMio 



Students X Part 
Students X Sg 
Students X SJq 



df 



MS 

55.266 
260.128 
0.715 
66.106 

237.979 
7.355 



5.558 
72.535 
1.101 
3.057 
19.454 



532.160 
79.618 

133.745 
29.441 
0.258 



57.327 
62.052 
29.588 



<1 
3.93 
<1 



4.15* 
<1 



<1 
1.17 
<1 
<1 
<1 



17.99** 
2.69 
4.52* 
<1 
Cl 



